azam moghadam; mohammadreza falsafinejhad; norali farokhi; masoomeh estaji
Abstract
Traditional approaches in educational measurement have some practical and theoretical challenges in demonstrating language competencies and their abilities in assessment candidates' skills and selecting them have been questioned. In order to overcome these restrictions cognitive diagnostic models (CDMs) ...
Read More
Traditional approaches in educational measurement have some practical and theoretical challenges in demonstrating language competencies and their abilities in assessment candidates' skills and selecting them have been questioned. In order to overcome these restrictions cognitive diagnostic models (CDMs) have been introduced and applied. Objective: The purpose of this study was diagnostic analysis of reading comprehension items of a general English language test (PhD entrance exam) to investigate underlying skills of a given test, inspection of model convergence and its fit, diagnostic power of the test and the mastery status of examinees. Method: The study conducted in cognitive diagnostic modeling. The population was all PhD candidates which majored in English teaching, linguistics, translation, and English literature. 2754 examinees were used as a sample. Task analysis, coding and verbal reports were applied to determine underlying skills of the test. Results: In qualitative section, 6 skills including using vocabulary knowledge, using syntactic knowledge, extracting explicit information or scan, drawing inference, connecting and synthesizing and using pragmatic knowledge were investigated. Also, quantitative analyses using non-compensatory reduced fusion model (FM) based on a Monte Carlo Markov chain (MCMC) indicated MCMC convergence and model fit and possibility of application of fusion model in English language's tests. The ability parameters were low for all skills. Using vocabulary knowledge was the simplest skill. The mean of item proportion-correct scores was .42 and the test did not have a high diagnostic power. Discussion and conclusion: Using cognitive diagnostic models in general and fusion model in particular results in achieving more information about tests and examinees' responses and it helps to reach the goal of assessment for learning and classify examinees as masters or non-masters correctly. Key Words: Cognitive Diagnostic Models (CDMs), Non-compensatory Fusion Model (FM), Reading Comprehension's Skills, and English Language
zahra naghsh; azam moghadam
Volume 2, Issue 8 , July 2012, , Pages 133-154
Abstract
In traditional approaches, single-level statistical models were generally used to analyze IEA data. In hierarchical linear models, each level is, however, introduced by its sub-model, and the variables' interrelations are explained in each specified level. The way the variables affect the existent relations ...
Read More
In traditional approaches, single-level statistical models were generally used to analyze IEA data. In hierarchical linear models, each level is, however, introduced by its sub-model, and the variables' interrelations are explained in each specified level. The way the variables affect the existent relations in other levels is also determined. The main purpose of this paper is to compare multi-level modeling and single-level analysis techniques and underline the importance of applying the former in analyzing the data extracted from the TIMSS 2007 questionnaires completed by the eighth graders. Due to their nature, the IEA data were analyzed by HLM software as the students were nested within classes, classes within schools, and schools within countries. In the single-level analysis, there was a significant relationship between self- concept, attitude and evaluation at 0.001 level with mathematics achievement (0.48, 0.296 and 0.134, respectively). Furthermore, the results of two-level analysis by one-way ANOVA with random effects showed that these three variables (self-concept, attitude and evaluation) explained 30.10% and 47% of mathematics achievement variance at student and school levels, respectively. The different results of these two analyses demonstrated the importance of using multi-level analyses for nested data like TIMSS. Regarding the nested nature of TIMSS data and the multi-level method used to extract them, the application of multi-level modeling techniques is recommended to obtain more detailed data on the factors influencing the students' achievement.
